A variety of ways exist for persons to communicate with each other using textual messages, including text messages (“texting”), instant messages, emails, etc. In large scale open (unrestricted) and closed (restricted) systems that enable text-based interactions between users, the potential exists for some participants to target other members in undesirable manners, including harassment, threats, or predatory behavior. These types of interactions represent a clear risk to involved minors and could represent a legal and reputational risk to the entities that manage systems and services that support communications by textual messages.
Identifying this type of activity across a large-scale, global service or system that has tens of millions of members in an accurate manner is extremely challenging. Simple keyword searches of textual communications result in vast amounts of “false positives” and/or erroneous data that is extremely difficult to analyze successfully.